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DEPTH MAP COMPRESSION TECHNIQUE 
FIELD OF THE INVENTION 

The present invention is directed towards a method of compressing depth 
maps, and in particular a method of representing depth maps in terms of curves, 
5 such as bezier curves, and ramp functions that is well suited for real time, or semi 
real time processing. 
BACKGROUND ART 

In converting 2D images into left and right eye images for stereoscopic 
viewing it is known to create depth maps to assist in the transmission and 
10 creation of the 3D image. Generally speaking, the creation of a depth map refers 
to a technique whereby each object in a scene is allocated a unique attribute 
(typically a shade of grey) depending upon the relative, or absolute, distance from 
the object to a reference point, for example the camera lens. 

For systems that seek to create stereoscopic images from a 2D image, the 
15 creation of this depth map is in most cases, if not all, an interim step in this 
conversion process. The operator, or system, will analyse a 2D image, create a 
unique depth map, and then finalise the process by creating left and right eye 
images. Depending on the circumstances, this final process may take place 
some time after the creation of the depth map. 
20 There presently exists a number of systems which attempt to convert 2D 

images into stereoscopic images. Whilst each of these systems may effectively 
create a depth map, the processes of obtaining those depth maps, and similarly, 
the process by which those depth maps are utilised differ. Further, in order to 
determine the depths of an object within an image and thereby a depth map, a 
25 number of techniques may be used, including the use of multiple cameras, laser 
range finders, radar imaging, and techniques using modulated radiation sources 
coupled with reflected radiation intensity detectors. 

For example, in the Applicants prior application PCT/AU96/00820, the 
contents of which are incorporated herein by reference, there was disclosed a 
30 number of techniques, including determining the distance of objects from a 
camera using variable focus techniques. Alternatively, the use of two cameras 
* and, an autocorrelator to determine the distance of objects from a camera was 
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also disclosed. 

As a result of these various techniques, the depth maps may be in various 
formats. The more common formats including, greyscale images, colour encoded 
depth images, or a floating point distance matrix. 
5 Whilst numerous techniques exist to convert 2D images to stereoscopic 

images, and in the process create depth maps, to date it has not been possible to 
combine these processes, such that one technique is utilised to create the depth 
map, and a different technique used to produce the stereoscopic image. That is, 
merging of the various techniques has not been possible, as existing systems are 
10 not able to process a depth map produced by a different process. 

The inability to combine processes can lead to the same 2D image being 
processed by a number of different techniques, thereby producing respective 
depth maps. The task of analysing a 2D image for conversion to a depth map 
can be complicated and in some cases time consuming, and it would be 
15 preferable to avoid the need to repeat this task depending on the overall 2D to 3D 
conversion process selected. 
OBJECTS OF THE INVENTION 

It is therefore an object of the present invention to provide a method of 
compressing depth maps, and in particular it is an object of the present invention 
20 to provide a relatively simple technique for representing depth maps in terms of 
curves, such as bezier curves, and ramp functions that is well suited for real time, 
or semi real time conversion. 
SUMMARY OF THE INVENTION 

With the above objects in mind, the present invention provides in one 
25 aspect a method of compressing depth maps including: 

determining the boundary of at least one object within a depth map; 
applying a curve to the boundary of each said at least one object; 
converting the continuous depth data within an area bounded by said 
curve into at least one ramp function. 
30 In a further aspect the present invention provides a method of compressing 

depth maps including the steps of: 

identifying at least one object within a depth map; 
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determining the outline of each said at least one object; and 

determining at least one ramp function to represent the depth of each said 
at least one object: 

In the preferred embodiment the curve used will be a bezier curve. 
5 BRIEF DESCRIPTION OF DRAWINGS 

To provide a better understanding of the present invention, reference is 
made to the accompanying drawings, which illustrate a preferred embodiment of 
the present invention. 

In the drawings: 

10 Figure 1 shows a representation of a depth map depicting three objects. 

Figure 2 shows how the edges of the three objects of Figure 1 may be 
detected. 

Figure 3 shows how the outline of the three objects of Figure 2 may be 
represented using bezier curves. 

1 5 DETAILED DESCRIPTION 

The Applicants have in prior applications AU 10884/97, PCT/AU98/01005, 
and Australian Provisional PQ1197, the contents all of which are herein 
incorporated by reference, disclosed various techniques used in the conversion of 
2D images to stereoscopic images. These techniques in part disclosed the 

20 creation of depth maps and the encoding of those depth maps. However, these 
techniques only considered the use of depth maps created as part of the 
respective process. They did not deal with a depth map created by a different 
process. 

Accordingly, if we assume that a depth map has been created either 
25 singularly, or as part of a conversion process, and that that depth map has been 
transmitted, retained or recorded in some way, then the present invention can be 
adopted to convert the depth map for transmission and/or further processing so 
as to display stereoscopic images. 

Referring now to Figure 1, there is shown by way of example a single 
30 video frame of a depth map representative of a 2D image. For purposes of 
explanation only, assume the video image has been digitised at 800 x 600 pixels 
with 8 bits of depth resolution thus allowing a possible 256 discrete depth levels. 
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Figure 1, as shown, contains three objects, a disk identified as object 1, a triangle 
identified as object 2 and an oblong identified as object 3. Each of these three 
objects is located a certain distance from the video camera. This distance is 
conveniently represented in the depth map by the shade of grey it is colored 
5 (shown as cross-hatching in Figure 1), normally the lighter the grey the closer the 
object is to the video camera. In this example object 1 is located closest to the 
camera and has a distance d1 from the viewer, and objects 2 and 3 distances d2 
and d3 respectively, with object 3 being located furthest from the camera. 

In order to convert the depth map, the objects within the depth map are 

10 first identified. That is, in the present example, objects 1, 2 and 3, are each 
identified as distinct objects. Then, once the objects have been identified, edge 
detection techniques can be utilised to determine the outline of each object. 

Figure 1 is also illustrative of a depth map produced, for example, in real 
time, from a range finder. In such cases, whilst humans are capable of seeing 

15 the outline of each object, the processor is unable to distinguish what each shape 
represents. Accordingly, the individual objects will not be known as all we have is 
a 2D image that represents the depth of each object in the image by a different 
shade of grey. 

In order to determine an object edge, or segment edge, conventional edge 
20 detection methods can be applied that will be familiar to those skilled in the art 
and include, although not limited to, Sobel, Marr-Hitdreth, Canny, zero crossing in 
derivatives and local energy/phase congruency techniques. Using such edge 
detection techniques the outline of each object can be detected as is illustrated in 
Figure 2. 

25 Once the outline of each object has been identified, then to aid in further 

processing it is possible to represent each of the outlines using bezier curves. It 
will be understood that other curves could also be used, although beziers are 
preferred due to the amount of data they require to represent the outline of an 
object. In order to achieve this standard curve matching techniques, known to 

30 those skilled in the art, can be applied to the outlines to convert them into bezier 
curves, as is illustrated in Figure 3. The application of a bezier curve to the outline 
of an object within a depth map, which can be applied manually, semi- 
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automatically or automatically, relies on the continuous nature of a depth map. 
That is, it is assumed that depth varies continuously within any segment of an 
object. 

If there is a depth discontinuity, then an object edge or object segment 
5 edge is present. That is, a further object is present, and should be identified as 
such. 

By this process the objects within the depth map are identified and 
conveniently the boundaries are represented as bezier curves. It is then 
necessary to account for the various depths of each of the objects. 

10 Once the outline of each object has been determined and represented in a 

convenient format, preferably bezier curves, it is necessary to represent the 
continuous depth value within the boundaries by one or a plurality of ramp 
functions. The Applicant* s have found that ramp functions are a very efficient way 
of compressing the depth data. For example, rather than store all the points 

15 along a linear depth ramp between points A and B as A1, A2, A3, ....B, it is 
possible to set the individual depths at A and B and assume a linear change of 
depth between the two points. The same approach can be applied for other 
shapes of depth - in general a mathematical expression that describes the depth 
function is used rather than the actual depths - thereby forming a highly efficient 

20 form of compression. Such ramp functions include, but are not limited to, linear, 
exponential, square law and radial. 

There are a number of methods to select the appropriate ramp functions 
which will be known to those skilled in the art. Such techniques, applied within 
the boundaries of each object, could include, fitting selected points of the depth 

25 data to the set of ramp functions and minimising the error values. Alternatively, 
the ramp functions may be determined by testing for constant areas and zero 
crossings in the zero, first and second order derivatives of the depth data. This 
will reveal flat, linear and curved areas respectively. 

It is desirable to determine the lowest number of ramp functions that can 

30 be used to represent the continuous depth value within the object. In order to 
determine the lowest number of ramp functions it is possible to compare the 
depth function with a library of pre-existing depth functions (linear, radial etc.,) 
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and find a best fit. For example, a least squares fit could be used to determine 
the most appropriate function in the library. 

Since the objects in the depth map can be represented by bezier curves 
and ramp functions this data may be represented, encoded and compressed 
5 using the techniques described in the Applicants previous disclosures 
PCT/AU98/01005 and PQ1197. In this way depth maps created by various 
techniques can be converted and utilised to create stereoscopic images without 
the need to analyse the original 2D image to create a unique depth map. 

In the past bezier curves have been used to create a depth map where 

10 one has not already existed. However, it has not been considered to convert a 
depth map to a bezier curve and object depth. By representing a depth map as a 
series of bezier points and object depths, a very efficient way of compressing a 
depth map can be addressed. By very efficiently compressing the depth map it 
can be added into the original 2D image and transmitted along with it. Since it is 

1 5 highly compressed it takes a little extra bandwidth and can thus be transmitted via 
existing video and Internet systems. 

It will be appreciated that the preceding process is well suited to fully 
automatic implementation in either hardware, software or a combination of both. 
This would enable live depth maps captured from a suitable device to be 

20 converted and encoded in a suitable format in real time for subsequent 
broadcasting or recording. 

It will be appreciated that the preceding process could be undertaken 
completely manually. In this embodiment, an operator would manually select the 
outline of each object and describe the bezier curve. Similarly, the operator may 

25 select a ramp function from a predetermined library of functions and/or create 
new ramp functions as required. 

It will also be appreciated that a semi automatic process could be 
implemented whereby the performance of the process is monitored by an 
operator and assistance is given by the operator if and when the process is 

30 unable to automatically determine the outline of an object or select an appropriate 
ramp function. 
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In an alternative embodiment, in order to represent the depth within an 
area bounded by a bezier curve, the ramp function may be replaced with other 
methods know to those skilled in the art, which include, but are not limited to, 
Meshes, Metaballs (Blobs), NURBS (Non Uniform Rational B-Splines), CSG 
(Constructive Solid Geometry) and TIN'S (Triangulated Irregular Networks). 

Modifications and variations to the conversion technique of the present 
invention may be apparent to one skilled in the art upon reading of this disclosure 
and such modifications and variations form part of the scope of the present 
invention. 
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THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS: 

1 . A method of compressing depth maps including: 
determining the boundary of at least one object within a depth map; 
applying a curve to the boundary of each said at least one object; 

converting the continuous depth data within an area bounded by said curve into 
at least one ramp function. 

2. A method as claimed in claim 1 , wherein said curve is a bezier curve. 

3. A method as claimed in claim 1 or claim 2, wherein if a depth discontinuity 
is detected during processing, the processing is suspended to allow the additional 
boundaries to be detected. 

4. A method as claimed in any preceding claim further including assigning a 
depth to each said at least one object. 

5. A method as claimed in any preceding claim wherein said ramp function is 
represented by at least one algorithm. 

6. A method as claimed in claim 5 wherein said ramp function includes at 
least one of linear, exponential, square law or radial. 

7. A method as claimed in any claim in any preceding claim wherein said 
ramp function is selected from a library of preexisting ramp functions. 

8. A method as claimed in claim 7 wherein at least squares fit is used to 
determine the ramp function selected from the library. 

9. A method as claimed in any preceding claim further including encoding the 
compressed data. 
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10. A method of compressing depth maps including the steps of: 
identifying at least one object within a depth map; 
determining the outline of each said at least one object; and 

determining at least one ramp function to represent the depth of each said 
at least one object. 

11. A method as claimed in claim 10 further including the step of determining 
at least one curve to represent said outline. 

12. A method as claimed in claim 1 1 , wherein said curve is a bezier curve. 

13. A method as claimed in any one of claims 10 to 12, wherein if a depth 
discontinuity is detected during processing further object identification is 
undertaken to determine additional objects. 

14. A method as claimed in any one of claims 10 to 13 further including the 
step of assigning a depth to each said at least one object. 

15. A method as claimed in any one of claims 10 to 14 wherein said ramp 
function is represented by at least one algorithm. 

16. A method as claimed in claim 15 wherein said ramp function includes at 
least one of linear, exponential, square law or radial. 

17. A method as claimed in any one of claims 10 to 16 wherein said ramp 
function is selected from a library of preexisting ramp functions. 

18. A method as claimed in claim 17 wherein a least squares fit is used to 
determine the ramp function selected from the library. 
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19. A method as claimed in any one of claims 10 to 18 further including 
encoding the compressed data. 

20. A method as claimed in claim 9 or claim 19 wherein said encoded data is 
combined with the depth map or an original image from which the depth map was 
derived. 



21 . A method as claimed in any preceding claim wherein said ramp function is 
replaced with meshes, metaballs, NURBS, CSG, or TIN'S. 

22. A method substantially as hereinbefore described with reference to the 
accompanying drawings. 
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